Computer simulations of language change notes

This website collects my personal notes on Computer simulations of language change. These notes are provided to bring full transparency to my research process. Of course, since they are only notes, they do not reflect my final thoughts on a topic, and should not be interpreted as such. To read finished papers, please consult my website. Do not use these notes as a basis for your own scientific research. Start from high-quality, peer-reviewed scientific literature instead.

From Usage to Grammar: The Mind’s Response to Repetition

(p. 2)

Usage-based grammar

separation

the use of language ⟷ internalised structure of language
≈ langue and parole (De Saussure)
≈ competence and performance (Chomsky)

historically

much emphasis on competence, not the actual use of language
but also: functionalists (study language use) + cognitive linguists
- joined under “usage-based” linguists

grammar from a usage-based POV

language is the cognitive organisation of one’s experience with language
e.g. frequency of use → conventionalisation
grammar (cognitive representation) arises from a categorising experience from language use

Converging trends in linguistic theory

methodological trends

theoretical issues are now commonly addressed through text corpora

(p. 3)

grammaticalisation

played a central role in emphasizing the point that both grammatical meaning and grammatical form come into being through repeated instances of language use
⇒ how grammar is created over time, thus setting a higher goal for linguistic explanation

Findings

conventionalised word sequences

characterises both written and spoken discourse
both formulaic language and idioms, but also conventionalised collocations

↓

idioms

conventionalized word sequences that usually contain ordinary words and predictable morpho-syntax
but: have extended meaning (usually of a metaphorical nature)
acknowledged to need lexical representation because of the unpredictable aspects of their meaning
- but: not completely isolated since many aspects of their meaning and form derive from more general constructions and the meaning of the component words in other contexts (nunberg et al. 1994)
e.g. pull strings, level playing field, too many irons in the fire

(p. 4)

idioms provide evidence for organized storage in which sequences of words can have lexical representation while still being associated with other occurrences of the same words, as in this diagram from bybee 1998.

prefabs

word sequences that are conventionalized, but predictable in all other ways
e.g. prominent role, mixed message, beyond repair, to need help
e.g. finish up, burn down, interested in, think of, think about

Erman and Warren 2000 found that what they call prefabricated word combinations constitute about 55% of both spoken and written discourse.

Speakers recognize prefabs as familiar, which indicates that these sequences of words must have memory storage despite being largely predictable in form and meaning.

The line between idiom and prefab is not always clear since many prefabs require a metaphorical stretch for their interpretation. The following may be intermediate examples, where at least one of the words requires a more abstract interpretation: break a habit, change hands, take charge of, give (s.o.) plenty of time, drive (s.o.) crazy. The lack of a clear boundary between idioms and prefabs would also suggest that both types of expression need memory storage.

grammar emergent from experience (Hopper, 1987)

mutable, and ever coming into being rather than static, categorical, and fixed
⇒ language is a complex dynamic system
- no structure a priori, but rather the apparent structure emerges from the repetition of many local events (in this case speech events)

(p. 5)

Goals of the article

Frequency effects on processing and storage

Three effects of token frequency that have been established in “recent” literature

1. reducing effect

high frequency words and phrases undergo phonetic reduction at a faster rate than low and mid frequency sequences
applies to phrases of extreme high frequency
e.g. i don’t know

↳ explanation

the articulatory representation of words and sequences of words are made up of neuromotor routines
when sequences of neuromotor routines are repeated, their execution becomes more fluent
⇒ increased fluency is the result of representing the repeated sequence at a higher level as a single unit

2. conserving effect

relates to the morphosyntactic structure of a string
high frequency sequences become more entrenched in their morpho-syntactic structure and resist change on the basis of more productive patterns
frequency strengthens the memory representations of words or phrases making them easier to access whole and thus less likely to be subject to analogical reformation
- applies to syntactic sequences as well
e.g. English irregular verbs → low frequency irregular verbs have a higher tendency to regularise

(p. 6)

3. autonomy effect

morphologically complex forms (or strings of words) of high frequency can lose their internal structure as they become autonomous from etymologically related forms
e.g. words with derivational affixes become less transparently related to their base forms as they become more frequent
e.g. the semantic opacity of words such as dislocate

This effect also applies in grammaticization when sequences that are originally complex (such as be going to) lose their semantic and syntactic transparency and move away from other instances of the words be, go, and to.

Effects and their corresponding frequencies
effect	frequency
reducing effect	high frequency items (linear relation)
conserving effect	high frequency items
autonomy effect	extremely high frequency items

The impossibility at the moment of specifying ranges for extreme high, medium and low is only a function of the state of our knowledge. As more empirical studies appear, absolute frequency ranges for each phenomena will eventually be specifiable.

Construction-based representations

representation based on constructions

turns out to be highly effective!
👁 ↓

“Cognitive representations of grammar are organized into constructions which are partially schematic, conventionalized sequences of morphemes with a direct semantic representation” (Goldberg 2003).

(p. 7)

constructions according to Goldberg

👁 ↓

↓ most lexically explicit

idioms with fixed lexical content: go great guns
idioms that are partially filled: jog <someone’s> memory
constructions with some fixed material: he made his way through the crowd
fully abstract constructions: they gave him an award

↓ most schematic

Almost all constructions contain some explicit morphological material, tying them fairly concretely to specific words or morphemes (e.g. way and the possessive pronoun in 3). The ditransitive construction in 4 contains no specific phonological material that identifies it as the ditransitive. Only the word order signals this. However, it should be noted that only a small class of verbs can occur in this construction so that it also has a grounding in lexical items.

↳ high use of prefabs and idioms in natural speech

a good deal of production (and perception) refers to sequences of pre-specified lexical choices rather than to abstract grammar
⇒ a model that builds a grammar from specific instances of language use (such as an exemplar model or a connectionist model) seems appropriate

Exemplar representation

What is exemplar theory?

exemplar theory

every token of experience is classified and placed in a vast organizational network as a part of the decoding process
this matching process has an effect on the representations themselves → new tokens of experience are not decoded and then discarded, but rather they impact memory representations

↳ impacting memory representations

token of linguistic experience is identical to an existing exemplar → mapped onto that exemplar, strengthening it
token of linguistic experience is similar (but not identical) to an existing exemplar → stored near similar exemplars to constitute clusters or categories as a new exemplar

↳ ordering of exemplar clusters

can be hierarchical
set of exemplars that are judged to be similar phonetically and represent the same meaning are clustered together and are represented at a higher level as a word or phrase
constructions emerge when phrases that bear some formal similarity as well as some semantic coherence are stored close to one another

(p. 8)

Applying exemplar theory to language

Which factors help us understand how constructions came into being, and change over time?

Exemplar representations allow specific information about instances of use to be retained in representation
Exemplar representation provides a natural way to allow frequency of use to determine the strength of exemplars
Exemplar clusters are categories that exhibit prototype effects. They have members that are more or less central to the category, rather than categorical features

↓ how to apply to language?

1. probabilistic grammar representation

based on the experience of the language user

specific storage	distributed storage
storing the relatively low frequency phrase such as beige curtains as a unit	mapping the two words onto existing exemplars of these words

Only when a sequence is repeated will access to it as a unit rather than by its parts become more efficient.

2. human memory capacity is “quite large”

an important feature of linguistic experience is the regular repetition of phonological strings, words, and constructions
repeated experience causes you to notice details
memory decays, despite repeated experience

(p. 9)

3. reorganisation is possible

linguistic memories represented as exemplars can undergo considerable reorganization, particularly when change is ongoing in a language

Exemplar representation of constructions

grouping

exemplars of words or phrases that are similar on different dimensions are grouped together in cognitive storage
⇒ constructions can emerge from this

origin of constructions

an exemplar representation of a partially filled construction would have experienced tokens mapping onto the constant parts of the construction exactly, strengthening these parts
- the open slots would not match exactly
if there are similarities (in particular semantic similarities) among the items occurring in the open slot, a category for these items would begin to develop

Spanish quedarse + adjective ‘become ADJ’

\text { quedarse } \quad\left\{\begin{array}{ll} \underline{\text { tranquilo }} & \text { 'tranquil' } \\ \text { quieto } & \text { 'quiet, still' } \\ \text { inmóvil } & \text { 'still, immobile } \end{array}\right\}

(p. 10)

Effects of repetition on particular instances of constructions

Various degrees of effect, depending upon the extent of frequency

low levels of repetition lead to conventionalization only (as in prefabs and idioms)
higher levels of repetition can lead to the establishment of a new construction with its own categories
extreme high frequency leads to the grammaticization of the new construction and the creation of grammatical morphemes and changes in constituency

Grammaticalisation

grammaticalisation

the creation of a new grammatical morpheme and a new construction out of a particular instance of an old construction
an existing construction with specific lexical items in it becomes more frequent, changes in various ways, and becomes a new construction

be going to

late 16^th century: an exemplar of a general purpose construction sometimes used in the progressive

Don Alphonso,
With other gentlemen of good esteem,
Are journeying to salute the emperor
And to commend their service to his will. (Two Gentlemen of Verona I.3)
I was sending to use Lord Timon myself… (Timons of Athens II.2)

(Shakespeare)

↳ most frequent instance used go as the main verb → therefore it grammaticalised

(p. 11)

Case study: the effects of frequency on going to

1. reducing effect

going to is often phonologically reduced to gonna
happens when words and phrases are often repeated
going to is processed as a single unit → neuromotor routine gets simplified

2. autonomy effect

lost association with other instances of the verb go
“going to” now leads its own life, so to speak

(p. 12)

3. loss of specific meaning

no longer: specific movement in space
in many contexts originally carried the pragmatic inference of intention
repeated instances of the construction in this pragmatic context give rise to the new construction

4. reanalysis

solidification as a single unit
frequently used together, so becomes to be processed together as a unit
going to becomes one unit, both phonologically and syntactically

New constructions without grammaticalisation

Argument: particular instances of constructions that have been experienced by a speaker must be present in cognitive representation

idioms with specific metaphorical meaning

must have cognitive representation, even though they are still related to the lexemes and construction from which they arose

prefabs

conventionalized but more transparent in meaning than 👁 ↑
must also have a representation (because they are conventional) but that representation is associated with the representations for their component parts

↓

focus here

the creation of constructions that carry specific pragmatic implications
but: still maintain at least a surface resemblance to the construction from which they arose

(p. 13)

Diner: Waiter, what’s this fly doing in my soup?
Waiter: Why, madam, I believe that’s the backstroke. (From Fillmore and Kay 1994)

The joke shows the possible ambiguity of the highlighted sequence. As Fillmore and Kay point out, the usual interpretation of ‘what is X doing Y’ is one of surprise at incongruity accompanied by more than a hint of disapproval. Because it is syntactically indistinct from the construction from which it arose, it gives the clever waiter license to interpret it as a literal question about what the fly is doing.

The implication of disapproval (a subjective interpretation) must have come from multiple instances of use with this negative nuance.

In order to know that a certain implication has occurred frequently and is associated with a certain string of words, speakers must register the context and the implications from the very first exposure. They could not wait until they had heard the expression frequently in a certain context to register this in memory, because if they did not remember each time, they would not know that they had heard it before. Thus it seems clear that in order for this construction to acquire its special meaning, the learner/hearer must record in memory the implication of incongruity and disapproval from the very first exposure. Because this construction was earlier just a special instance of a more general construction, the changes it has undergone indicate that particular instances of constructions are registered in linguistic memory indexed with their implications and contexts of use.

(p. 14)

(p. 15)

Special phonological reduction

special phonology through frequency of use

e.g. I don’t know, I don’t think
phonological reduction in excess of that which could be attributed to online reduction processes
⇒ reduction has accumulated in representation

high frequency words and phrases

have larger exemplar clusters and greater ranges of variation than low frequency words and phrases

As phonological reduction occurs on line, high frequency words and phrases have more opportunity to undergo reduction. When an already reduced exemplar is selected for production, it may undergo further reduction, leading to more advanced reduction in high frequency phrases .

Exemplar representation of the linguistic sign

We know that specific phonetic representations are associated in representation with specific meanings and contexts because in grammaticization phonological reduction only occurs in the grammaticizing construction. Thus the form gonna is associated with the intention/future meaning and not with the movement in space or purpose meaning.

(p. 16)

Four exemplars with phonetic shape associated uniquely with meaning and context

Variable associations of form and meaning in a linguistic sign

(p. 17)

(p. 18)

Frequency effects: category formation

Nature of the category formed by the lexical items that are used in the open slot in constructions

what can go in the slot of a construction?

sometimes: highly generalized → takes all nouns or all verbs of the language
other times: semantically constrained → only specific nouns of verbs are allowed

Bybee and Eddington found evidence that high frequency lexical instances of constructions can act as the central members of the category formed by members of the open slot in a construction.

(p. 19)

These adjectives form a category with proto-type effects: the most frequent member is central and the other members are more marginal. Other members included adjectives that are basically synonymous with inmóvil such as parado ‘stopped, standing’; adjectives that are figurative/metaphorical de piedra ‘of stone’; and adjectives that share the ‘motionless’ feature but add other features, such as atrapado ‘trapped. Thus we argue for a category with the high frequency member as the center of the category.

Rather than attempting to find abstract semantic features that characterize all the adjectives used with a particular verb, Bybee and Eddington propose that the verbs quedarse and ponerse each have a number of such clusters of adjectives. For instance, central members of categories for quedarse besides the two mentioned above are the following, which are related to quedarse inmóvil-- quedarse quieto ‘to become still’, quedarse tranquilo ‘to calm down’, quedarse callado ‘to quiet down, become silent’, quedarse dormido ‘to fall asleep’ and two other categories that are unrelated, quedarse sorprendido ‘to be surprised’ and quedarse embarazada ‘to get pregnant’. See Bybee and Eddington for further details.

Evidence hat the most frequent member is central to the category

the larger, more productive categories (that is, those with the highest type frequency) are organized around a frequent member, suggesting that productive uses of the verb occur on the basis of reference to semantic similarity to a frequent member
- The groups of adjectives that occurred with the verbs that were less productive did not have a high frequency member.
- Of the four verbs studied, only two, quedarse and ponerse, showed categories organized around high frequency exemplars. The other two verbs, which are much less frequent with animate subjects, had a much more scattered and miscellaneous distribution with adjectives.
Family resemblance structure uses the most frequent adjective as central.
We conducted an experiment on the acceptability of verb + adjective combinations. We asked 48 peninsular Spanish speakers to rate the acceptability of sentences we took from the corpus. All stimuli were all naturally-occurring utterances of Spanish. The subjects were asked to rate the stimuli on a 5 point scale from ‘perfectamente bien’ ‘perfectly fine’ to ‘raro’ or ‘strange/rare’.

(p. 20)

three groups of stimuli

👁 ↓

High frequency phrases (quedarse inmóvil)
Low frequency phrases with a close semantic affinity to a high frequency phrase (quedarse parado)
Low frequency phrases with no semantic connection to a high frequency phrase (quedarse orgullosísimo ‘become very proud’)

Results for quedarse

Acceptability judgments. ‘1’ is high acceptability and ‘5’ is low.

↓

The high frequency phrases were judged most acceptable. (X² (1) = 51.4, p < .0001)
A significant difference was found between the high frequency phrases and the low frequency phrases that were semantically similar to the high frequency phrases. (For quedarse and ponerse together: X² (1) = 6.22, p < .013.)
The strongest result was the significant difference between the low frequency related items and the low frequency unrelated items. (X² (1) = 32.9, p < 0.0001)

↳ both frequency and semantic similarity to a frequent exemplar significantly influenced the subjects’ judgments of acceptability
↳ not only do speakers record specific exemplars of constructions in memory, but that frequency of use also has an impact on the strength of these exemplars and on category formation

(p. 21)

Frequency effects: resistance to change

morpho-syntactic constructions

resistant to change if they are highly frequent
constructions with odd or irregular properties resist change in the particular exemplars that are of high frequency

The use of not-negation and no-negation (also called neg-incorporation) as in the examples such as the following: (Tottie 1991)

He did not see anything. (not-negation)
He saw nothing. (no-negation)

Diachronically, the no-negation construction predates the construction with not and the more recently developed construction with not is more productive, gradually increasing its usage. Thus we can predict that the no-negation construction would be maintained primarily in high frequency constructions and collocations, while notnegation would be spreading to more contexts.

Tottie studied these two constructions in a large corpus of spoken and written British English. She extracted only those examples where the use of the alternate construction would have the same meaning and implications (as in 22 and 23). She found that certain constructions, especially existential be (as in example 24), stative have (as in 25) and copular be (as in 26) have a higher use of no-negation than lexical verbs do, as shown in Table 2. This suggests that no-negation, rather than being an option for all sentences, has become associated with certain constructions.

	Spoken		Written
existential be	34/38	89%	96/98	98%
stative have	18/28	64%	41/42	98%
copular be	12/20	60%	26/47	55%
lexical verbs	20/76	26%	67/104	64%

These three constructions are fairly frequent, accounting together for more of the data than all the lexical verbs combined. Their frequency could help explain the fact that they preserve the older construction; much like the old ablauting verbs of English (break, broke; write, wrote; etc.) their high frequency strengthens their representations and makes them less likely to be reformed on the more productive pattern. This suggests that a frequency effect might also be found among the lexical verbs. In fact, certain frequent verbs, i.e. know, do, give, make account for many of the examples of no-negation in the lexical examples.

(p. 22)

Conclusion

Speakers are familiar with certain word combinations (prefabs) which are in no way exceptional in meaning or form.
Special phonological reduction accrues to specific high frequency phrases.
New constructions are created out of specific instances of old general constructions.
In grammaticization, changes in phonology, semantics, and structure occur in extremely high frequency constructions.
Certain higher frequency exemplars of constructions dominate formation of categories of items within constructions.
High frequency exemplars of constructions resist change on the basis of more productive constructions

Computer simulations of language change notes